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DETAILED ACTION 

1 . This communication is in response to Remarks filed 09/09/2005. 

2. Claims 1-32 are pending in the application. Claims 1, 3-5, 7, 10-12, 15, 
17-19, 21, 24-28 have been amended. Claims 29-32 have been added. 

Response to Arguments 

3. Applicant's arguments with respect to claims 1-32 have been considered 
but are moot in view of the new ground(s) of rejection. 

Claim Rejections - 35 USC §112 

4. The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

5. Claims 29-32 are rejected under 35 U.S.C. 112, second paragraph, as 
being indefinite for failing to particularly point out and distinctly claim the subject 
matter which applicant regards as the invention. The applicant states in claims 1 
and 1 5 that the output is a non-language graphic image, and then further states 
the non-language graphic image comprises a language component consisting of 
letters, numbers and words. This is a contradiction; if a graphic image contains a 
language component it cannot be purely a non-language graphic image, as the 
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meaning of the output can be derived from the language component, making the 
output dependent on the language component as well as the image. 

Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for 
all obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described 
as set forth in section 1 02 of this title, if the differences between the subject matter sought to 
be patented and the prior art are such that the subject matter as a whole would have been 
obvious at the time the invention was made to a person having ordinary skill in the art to which 
said subject matter pertains. Patentability shall not be negatived by the manner in which the 
invention was made. 

7. Claims 1-32 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Sakiyama et al. (5,659,764) in view of Greene, Jr. et al. (6,377,925). 

As to claim 1 , Sakiyama et al. teach: 

a speech input responsive to verbal communication, (a voice interface unit 
for converting a signal sent from the microphone into a kana and kanji coexistent 
sentence recognizable by a computer, col. 8, lines 24-26); 

a speech recognition processor responsive to said speech input and 
creating an electronic output representing said verbal communication, (a voice 
recognition apparatus for creating a textual output from a voice input, col. 8, 
lines 26-28); 
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a database storing words and graphic image designators corresponding to 
said words, (sign language word CG pattern storage, allowing the input to be 
matched to sign language-like expression, col. 8, lines 38-41, and line 48); 

a processor responsive to said electronic output representing said verbal 
communication to access a graphic image designator from said database which 
represents said verbal communication, (a spoken language processing unit for 
performing a spoken language analysis of the inputted voice language 
composition, rearranging sign language words so that they may meet a sign 
language-like expression, col. 8, lines 37-41); and 

a graphic image generator responsive to said graphic image designator to 
generate a graphic image, which represents said verbal communication, (CG 
animation generation unit for supplying a graphic, matching the voice input, col. 
8, lines 38-41, and 46-48). 

Sakiyama et aL do not teach non-language graphic image designators. 

However, Greene, Jr. et al. teach an image or animation translator, where 
the animation can portray cartoons, and the images are related to each word or 
phrase, pictures of items, etc. For example, the word "cow" can be associated to 
an image of a cow, while the phrase "cow jumped over the moon" can be 
associated with an animation of a cow jumping over the moon. 

Therefore it would have been obvious to one of ordinary skill in the art at 
the time of the invention to combine the teachings of Sakiyama et al. with the 
methods of Greene, Jr. et al. to create an improved communication system for 
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the hearing impaired, by associating images and animations with the input to be 
communicated, as taught by Greene Jr. et al. (col. 1, lines 19-32). 

As to claim 15, Sakiyama et al. teach: 

a text source for generating electronic output representing words (a kana 
and kanji coexistent sentence recognizable on a computer, realizable by a voice 
recognition apparatus, col. 8, lines 27-29); 

a database storing words and graphic image designators corresponding to 
said words (a CG pattern storage in accordance with an arrangement of sign 
language words, with the sign language animations, col. 8, lines 47-50), 

a processor responsive to said electronic output representing words to a 
access a graphic image designator from said database which represents said 
words (a CG animation generation unit applies a sign language animation to the 
corresponding sign language word denoted by the spoken language processing 
unit, coL 8, lines 46-48); and 

a graphic image generator responsive to said graphic image designator to 
generate a graphic image which represents said words (a CG animation 
generation unit applies a sign language animation to the corresponding sign 
language word denoted by the spoken language processing unit, col. 8, lines 46- 
48). 

Sakiyama et al. do not teach non-language graphic image designators. 
However, Greene, Jr. et al. teach an image or animation translator, where 
the animation can portray cartoons, and the images are related to each word or 
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phrase, pictures of items, etc. For example, the word "cow" can be associated to 
an image of a cow, while the phrase "cow jumped over the moon" can be 
associated with an animation of a cow jumping over the moon. 

Therefore it would have been obvious to one of ordinary skill in the art at 
the time of the invention to combine the teachings of Sakiyama et al. with the 
methods of Greene, Jr. et al. to create an improved communication system for 
the hearing impaired, by associating images and animations with the input to be 
communicated, as taught by Greene Jr. et al. (col. 1 , lines 19-32). 

As to claims 2 and 16, Sakiyama et al. teach at least one computing 
device, which includes said speech processor (computer with a voice recognizer, 
col. 8, lines 28-30). 

As to claims 3 and 17, Sakiyama et al. teach at least one computing 
device is selected from the group consisting of personal computers, workstations, 
servers, clients, mini-computers, main-frame computers, laptop computers, 
mobile computers, palm-top computers, hand-held computers, set top boxes for 
a television, web-enabled televisions, interactive kiosks, personal digital 
assistants, interactive wireless communication devices, web-enabled wireless 
communication devices, mobile web browsers, pagers and cellular phones, (a 
computer with a voice interface, (col. 8, lines 27-28). It would have been obvious 
to one of ordinary skill in the art at the time of the invention that this computer 
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could be any type of computer, including a personal computer, laptop computer, 
mobile computer, or client computer, to increase the flexibility of the system). 

As to claims 4 and 18, Sakiyama et al. teach at least one computing 
device comprises a display screen for displaying said graphic image (display 
screen, col. 8, lines 54-55). 

Sakiyama et al. do not teach non-language graphic image. 

However, Greene, Jr. et al. teach an image or animation translator, where 
the animation can portray cartoons, and the images are related to each word or 
phrase, pictures of items, etc. For example, the word "cow" can be associated to 
an image of a cow, while the phrase "cow jumped over the moon" can be 
associated with an animation of a cow jumping over the moon. 

Therefore it would have been obvious to one of ordinary skill in the art at 
the time of the invention to combine the teachings of Sakiyama et al. with the 
methods of Greene, Jr. et al. to create an improved communication system for 
the hearing impaired, by associating images and animations with the input to be 
communicated, as taught by Greene Jr. et al. (col. 1, lines 19-32). 

As to claims 5 and 19, Sakiyama et al. teach one computing device 
accessing a network (Fig. 22, depicts two computer devices linked together for 
communication, creating a network) 
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As to claims 6 and 20, Sakiyama et al. teach the speech recognition 
processor comprises an acoustic processor and a word decoder, (a voice 
interface unit for converting a voice input signal from a microphone into a kana 
and kanji coexistent sentence recognizable by a computer, col. 8, lines 25-28). 

As to claims 7 and 21, Sakiyama et al. teach the graphic image is selected 
from the group consisting of static pictures, moving pictures, and animations 
(computer graphic (CG) patterns) (assigning a CG patterns to the corresponding 
sing language word (col. 8, lines 38-40), where it would have been obvious to 
one of ordinary skill in the art at the time of the invention for the patterns to 
include, static pictures, moving pictures, and animations as communication 
involving sign language involves postures that are static and also moving, and as 
a result the static pictures, moving pictures, and animations would be needed to 
properly display the language). 

Sakiyama et al. do not teach non-language graphic image. 

However, Greene, Jr. et al. teach an image or animation translator, where 
the animation can portray cartoons, and the images are related to each word or 
phrase, pictures of items, etc. For example, the word "cow" can be associated to 
an image of a cow, while the phrase "cow jumped over the moon" can be 
associated with an animation of a cow jumping over the moon. 

Therefore it would have been obvious to one of ordinary skill in the art at 
the time of the invention to combine the teachings of Sakiyama et al. with the 
methods of Greene, Jr. et al. to create an improved communication system for 
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the hearing impaired, by associating images and animations with the input to be 
communicated, as taught by Greene Jr. et al. (col. 1, lines 19-32). 

As to claims 8 and 22, Sakiyama et al. teach the speech recognition 
processor comprises a syntax module (converting the voice input into kana and 
kanji coexistent sentences, and then using these sentences to link together the 
corresponding sign language animations, and rearranging sing language words 
so they meet a sign language-like expression, (col. 8, lines 26-28, and col. 10, 
39-42). It would have been obvious to one of ordinary skill in the art at time of 
the invention that as the words are being rearranged so that they may meet a 
sing language-like expression, they would be placed in the order that creates the 
best grammatical expression). 

As to claims 9 and 23, Sakiyama et al. teach the speech recognition 
processor comprises a phrase correlator, (a spoken language processor that 
performs analysis on the inputted voice language composition, finds each of the 
sign language words, and then creating sign language-like expression from the 
sign language words, col. 8, lines 37-44). It would have been obvious to one of 
ordinary skill in the art at the time of the invention that every phrase or grouping 
of sign language words that has the same meaning would have the same sign 
language-like expression created for it to increase the ability of the aurally 
handicapped person to speak with an aurally normal person). 
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As to claims 10 and 24, Sakiyama et al. teach: 

inputting verbal communication to a processor (a microphone for inputting 
words, col. 8, lines 24-26); 

matching, in said processor, said verbal communication with graphic, 
images representing said verbal communication (matching the verbal 
communication with an animation representing a sign language, col. 8, lines 38- 
41 , and lines 48-50); and 

outputting from said processor said graphic images (displaying the 
animations on a display device, col. 8, lines 52-55). 

Sakiyama et al. do not teach non-language graphic image. 

However, Greene, Jr. et al. teach an image or animation translator, where 
the animation can portray cartoons, and the images are related to each word or 
phrase, pictures of items, etc. For example, the word "cow" can be associated to 
an image of a cow, while the phrase "cow jumped over the moon" can be 
associated with an animation of a cow jumping over the moon. 

Therefore it would have been obvious to one of ordinary skill in the art at 
the time of the invention to combine the teachings of Sakiyama et al. with the 
methods of Greene, Jr. et al. to create an improved communication system for 
the hearing impaired, by associating images and animations with the input to be 
communicated, as taught by Greene Jr. et al. (col. 1, lines 19-32). 
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As to claims 1 1 , and 25, Sakiyama et al. teach transmitting said graphic 
images to a display screen (displaying the animations on a display device, col. 8, 
lines 52-55). 

Sakiyama et al. do not teach non-language graphic image. 

However, Greene, Jr. et al. teach an image or animation translator, where 
the animation can portray cartoons, and the images are related to each word or 
phrase, pictures of items, etc. For example, the word "cow" can be associated to 
an image of a cow, while the phrase "cow jumped over the moon" can be 
associated with an animation of a cow jumping over the moon. 

Therefore it would have been obvious to one of ordinary skill in the art at 
the time of the invention to combine the teachings of Sakiyama et al. with the 
methods of Greene, Jr. et al. to create an improved communication system for 
the hearing impaired, by associating images and animations with the input to be 
communicated, as taught by Greene Jr. et al. (col. 1, lines 19-32). 

As to claims 12 and 26, Sakiyama et al. do not teach responding to 
graphic images by inputting additional verbal communication (Fig. 22 depicts a 
communication network, where output is to both users through monitors, 
(elements 7 and 28), whereby when the aurally normal user receives a 
communication they are able to reply through the microphone, transmitting a 
message to be transformed and displayed to the aurally handicapped users). 

Sakiyama et al. do not teach non-language graphic image. 
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However, Greene, Jr. et al. teach an image or animation translator, where 
the animation can portray cartoons, and the images are related to each word or 
phrase, pictures of items, etc. For example, the word "cow" can be associated to 
an image of a cow, while the phrase "cow jumped over the moon" can be 
associated with an animation of a cow jumping over the moon. 

Therefore it would have been obvious to one of ordinary skill in the art at 
the time of the invention to combine the teachings of Sakiyama et al. with the 
methods of Greene, Jr. et al. to create an improved communication system for 
the hearing impaired, by associating images and animations with the input to be 
communicated, as taught by Greene Jr. et al. (col. 1, lines 19-32). 

As to claim 13, Sakiyama et al. teach inputting verbal communication 
comprises speaking (a microphone for inputting a voice, col. 8, lines 24-26). 

As to claims 14 and 28, Sakiyama et al. teach inputting verbal 
communication and said step of responding by inputting additional verbal 
communication are undertaken by different users, (Fig. 22 depicts a 
communication network, where output is to both users through monitors, 
(elements 7 and 28), whereby when the aurally normal user receives a 
communication they are able to reply through the microphone, transmitting a 
message to be transformed and displayed to the aurally handicapped users). 
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As to claim 27, Sakiyama et al. teach step of inputting words comprises 
using a keyboard, mouse, touch-pad, joystick, or pen-input-pad (a keyboard or 
motion sensor input, Fig. 1 A, elements 3 and 8). 

As to claims 29 and 31 , Sakiyama et al. do not teach the non-language 
graphic element image comprises a language component disposed on or about 
said non-language graphic image. 

However, Greene, Jr. et al. teach an image or animation translator, where 
the animation can portray cartoons, and the images are related to each word or 
phrase, pictures of items, etc. For example, the word "cow" can be associated to 
an image of a cow, while the phrase "cow jumped over the moon" can be 
associated with an animation of a cow jumping over the moon, where it would be 
obvious to one of ordinary skill in the art that since pictures of items can be used 
to represent the input words, pictures of signs would include a language element. 

Therefore it would have been obvious to one of ordinary skill in the art at 
the time of the invention to combine the teachings of Sakiyama et al. with the 
methods of Greene, Jr. et al. to create an improved communication system for 
the hearing impaired, by associating images and animations with the input to be 
communicated, as taught by Greene Jr. et al. (col. 1, lines 19-32). 

As to claims 30 and 32, Sakiyama et al. do not teach the language 
component is selected from the group consisting of letters, numbers, and words. 
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However, Greene, Jr. et al. teach an image or animation translator, where 
the animation can portray cartoons, and the images are related to each word or 
phrase, pictures of items, etc. For example, the word "cow" can be associated to 
an image of a cow, while the phrase "cow jumped over the moon" can be 
associated with an animation of a cow jumping over the moon, where it would be 
obvious to one of ordinary skill in the art that since pictures of items can be used 
to represent the input words, pictures of signs would contain letters, numbers, 
and words, further allowing the user to understand what the picture represents, 
since the signs would contain these elements in the real world. 

Therefore it would have been obvious to one of ordinary skill in the art at 
the time of the invention to combine the teachings of Sakiyama et al. with the 
methods of Greene, Jr. et al. to create an improved communication system for 
the hearing impaired, by associating images and animations with the input to be 
communicated, as taught by Greene Jr. et al. (col. 1, lines 19-32). 

Conclusion 

8. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. See PTO-892. 

9. Applicant's amendment necessitated the new ground(s) of rejection 
presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. 



Application/Control Number: 09/891,030 Page 
Art Unit: 2654 

See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as 
set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire 
THREE MONTHS from the mailing date of this action. In the event a first reply is 
filed within TWO MONTHS of the mailing date of this final action and the advisory 
action is not mailed until after the end of the THREE-MONTH shortened statutory 
period, then the shortened statutory period will expire on the date the advisory 
action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be 
calculated from the mailing date of the advisory action. In no event, however, will 
the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

10. Any inquiry concerning this communication or earlier communications from 
the examiner should be directed to Thomas E. Shortledge whose telephone 
number is (571 )272-7612. The examiner can normally be reached on M-F 8:00 - 
4:30. 

If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, Richemond Dorvil can be reached on (571)272-7602. 
The fax phone number for the organization where this application or proceeding 
is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from 
the Patent Application Information Retrieval (PAIR) system. Status information 
for published applications may be obtained from either Private PAIR or Public 
PAIR. Status information for unpublished applications is available through 
Private PAIR only. For more information about the PAIR system, see http://pair- 
direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll- 
free). 



TS 

11/23/2005 




